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Abstract 

To support the development and evaluation of 
future function allocation concepts for separation 
assurance systems for the Next Generation Air 
Transportation System, this paper presents the design 
and human-in-the-loop evaluation of three feasible 
function allocation concepts that allocate primary 
aircraft separation assurance responsibilities and 
workload to: 1) pilots; 2) air traffic controllers 
(ATC); and 3) automation. The design of these 
concepts also included rules of the road, separation 
assurance burdens for aircraft of different equipage 
levels, and utilization of advanced weather displays 
paired with advanced conflict detection and 
resolution automation. Results of the human-in-the- 
loop simulation show that: a) all the concepts are 
robust with respect to weather perturbation; b) 
concept 1 (pilots) had highest throughput, closest to 
assigned spacing, and fewest violations of speed and 
altitude restrictions; c) the energy of the aircraft 
during the descent phase was better managed in 
concepts 1 and 2 (pilots and ATC) than in concept 3 
(automation), in which the situation awareness of 
pilots and controllers was lowest, and workload of 
pilots was highest. The paper also discusses further 
development of these concepts and their 

augmentation and integration with future air traffic 
management tools and systems that are being 
considered for NextGen. 

I. Introduction 

The current National Airspace System (NAS) is 
under stress and air traffic density is expected to 
increase twofold to threefold by 2025. If not 
addressed, this trend will lead to billions of dollars of 
lost economic activity, increased safety risks, and 
decreased system reliability and stability. Managed 
by the Joint Planning and Development Office 
(JPDO), the Next Generation Air Transportation 
System (NextGen) is an initiative that addresses these 
issues through a comprehensive and ongoing 


transformation of the NAS with continuous 
deployment of improvements and updates 
implemented in stages between 2012 and 
2025. NextGeiTs goals are to increase the NAS' 
efficiency, capacity, and security, while maintaining 
or improving safety, and to reduce the environmental 
impact of aviation [1, 2]. 

To achieve these goals, new system operating 
principles and capabilities are being conceived and 
developed under NextGen. An important area that 
has received considerable research interest is the 
development of separation assurance concepts that 
involve significant changes to the roles and 
responsibilities of the air traffic controllers (ATCs), 
pilots, and ground-based and airborne automation 
systems. The concepts vary by the extent to which 
the separation assurance function is distributed 
among pilots, controllers, and automation. One of 
the main issues that these concepts address is the 
significant increase in the ATC workload that comes 
with increased air traffic densities. Automation, in 
particular, is a primary means by which designers are 
seeking to address this workload issue and thereby 
safely increase the allowable air traffic 
densities. Methods by which automation can be 
brought to bear on this problem may be defined with 
respect to the relative degree of human involvement 
in generating resolutions for conflicts. A number of 
current research efforts aim to develop automated 
systems which detect projected losses of separation 
(conflicts) and then generate conflict resolutions 
(trajectory modifications), thereby, substantially 
augmenting and/or replacing functions now 
performed by ATCs [3-6]. For instance, the ground- 
based automation tool developed under the Advanced 
Airspace Concept (AAC) by researchers at NASA 
Ames [7, 8] is designed to detect conflicts and then to 
generate and transmit conflict-free routes (via 
datacom) to appropriately equipped aircraft for 
execution. In this concept, separation assurance 
could be managed jointly by the ground based 
automation system and pilots, while controllers are 



mainly responsible for strategic management of air 
traffic flow and separation of unequipped 
aircraft. An alternative to the above approach has 
also been proposed in which the controller delegates 
the separation responsibility to the pilot. The 
consensus view in the NextGen community is that 
this approach would require advance airborne 
automation in the form of a cockpit situation display 
(CSD) and conflict detection and resolution (CD&R) 
tools for separation management [9-17]. 

While the benefits of automation have been 
shown, the exact roles and responsibilities of pilots, 
controllers, and automation surrounding these 
NextGen concepts are still in development. In 2009 a 
human-in-the-loop simulation was conducted to help 
support the development and evaluation of allocating 
separation assurance responsibilities and workload to 
these three entities. The study presented flights 
through and around weather during the cruise phase 
of flight, and for which the differing allocations were 
examined. Previous papers have reported subjective 
and performance data for the cruise phase in this 
study [18-19]. However, each of the flights in this 
study culminated with a final weather free arrival 
phase down to the meter fix during which ATCs were 
always responsible for separation management, but 
there has been only a limited examination of how the 
cruise phase functional allocation affected 
performance associated with the arrival phase. This 
paper presents a more detailed analysis of this aspect 
of the study. Section II describes the design of the 
three concepts using a human-centered approach, 
Section III describes the experimental design and 
methods of the human-in-the loop simulation used to 
evaluate these concepts, while Sections IV and V 
discuss this study’s results, their implications for 
further development of these concepts, and their 
augmentation and integration with future air traffic 
management tools and systems that are being 
considered for NextGen. 

II. Design of Concepts 

Three function allocation concepts were 
conceived and designed with a human-centered 
approach that distributes separation assurance 
responsibility among pilots, controllers, and 
air/ground automation. Specific divisions of 
functions were chosen to vary operator workload and 
situation awareness, and to examine system 
performance across concepts. A set of common 


assumptions regarding aircraft equipage, data 
communication technologies, airborne and ground- 
based automated decision support tools, and rules of 
the road for conflict management were developed. 
These assumptions include: 

1. There are two groups of aircraft operating in 
the airspace: a) 50% of all aircraft are trajectory flight 
rule (TFR) aircraft, and have a cockpit situation 
display (CSD) on board integrated with a route 
assessment (trajectory replanning) tool (RAT) and a 
3D-weather display [20] plus, in some cases, CD&R 
tools. Using the RAT, the pilot can manually make 
changes to the trajectory; and b) 50% of all aircraft 
are instrument flight rule aircraft (IFR) managed by 
ATC and do not have CD&R tools. In this study, all 
experimental pilots flew TFR aircraft. 

2. All aircraft have the capability to 
communicate and exchange information with ATC 
through Controller Pilot Data Link Communications 
(CPDLC) and Automatic Dependent Surveillance 
Broadcasting (ADS-B). 

3. Pilots of both TFR and IFR aircraft are 
responsible for interval management operations [15], 
using flight deck automation [11] to space 105 sec 
behind an assigned lead by the final approach fix. 

4. The ground and (when present) airborne 
auto-resolver systems use the NASA Advanced 
Airspace Concept (AAC) [7, 8] algorithm for 
detection and resolution of conflicts between 4 and 
12 minutes to predict loss of separation (LOS), and 
use the Tactical Separation Assisted Flight 
Environment (TSAFE) algorithm for avoidance of 
conflicts less than 4 minutes to LOS. The auto- 
resolver tools on the ground and in the flight deck do 
not take weather into account; thus, pilots must 
ensure all resolutions are weather free. 

5. To resolve a conflict, and if equipped, the 
TFR pilot can use either the airborne auto-resolver on 
board to generate a conflict-free resolution and check 
for weather-free, or the RAT to do the same tasks. 
Similarly, the controller can use either the ground- 
based auto-resolver to generate resolutions and check 
for weather-free, or the manual trial planning tool to 
do the same tasks. 

6. Rules of the road: When in conflict with IFR 
aircraft, operators or automation must resolve the 
conflict by changing the trajectory of the TFR 
aircraft. For conflicts that are 4 minutes or less to 



LOS, TSAFE provides conflict-free resolutions to the 
controller for both TFR and IFR aircraft. 

With these assumptions providing a baseline for 
the operation, three function allocation concepts were 
developed to vary the responsibility for conflict 
resolution as well as the amount of conflicts that 
pilots, controllers, and automation must manage. The 
three concepts are summarized below: 

Concept 1: Pilot Primary, Controller Secondary 

TFR pilots are equipped with a conflict probe 
tool that detects conflicts and alerts the operator, and 
a conflict resolution tool with which the operator can 
either request algorithmically-generated conflict 
resolutions or create their own conflict resolutions. 
TFR pilots are responsible for generating reroutes to 
avoid weather and for identifying and resolving TFR- 
TFR and TFR-IFR conflicts with ownship. IFR pilots 
are responsible only for verbally requesting reroutes 
around weather. Controllers are equipped with 
equivalent ground-based tools and responsible for 
IFR-IFR conflicts. Controllers managed all IFR 
aircraft. When avoiding traffic and weather, pilots 
self-modify and execute their routes and broadcast 
their route modifications via ADS-B, thereby 
updating other TFR aircraft and the air traffic control 
system. Pilots monitor voice frequencies but will 
only receive clearances from the controller if their 
spacing becomes discontinuous and responsibility is 
delegated to the controller to adjust. Through these 
allocations, pilots are responsible for resolving 75% 
of the total conflicts, ATC is responsible for 25% of 
the total conflicts, and the Auto-resolver agent is not 
responsible for resolving any conflicts. 

Concept 2: Controller Primary, Automation 

Secondary 

TFR pilots are equipped with a conflict probe 
tool and a conflict resolution tool, as they are in 
Concept 1, but they are not responsible for resolving 
any conflicts. Instead, they have the capability of 
generating conflict and weather free reroutes, and 
datalinking them to the controller, but the controller 
must approve/disapprove or offer alternatives to all 
such reroutes. Both IFR and TFR pilots are 
responsible for requesting reroutes around weather. 
Controllers are equipped with equivalent ground- 
based tools and are responsible for generating and/or 
approving TFR-IFR and IFR-IFR conflict 
resolutions. For TFR-IFR conflicts, the controller 


modifies the route of the IFR aircraft unless an 
acceptable TFR resolution is datalinked to the 
controller. An autoresolver agent, which acts 
independently when responsibility is delegated to it 
and datalinks route modifications directly to 
operators, is responsible for TFR-TFR conflicts, but 
does not take weather into account. While the 
controller is not in the decision-making loop for 
initial resolution of TFR-TFR conflicts, TFR pilots 
can review route modifications made by the 
autoresolver agent before executing them, and return 
them to the controller if not weather free. Through 
these allocations, pilots are not responsible for 
resolving any conflicts, ATC is responsible for 
resolving 75% of the total conflicts, and the auto- 
resolver agent is responsible for resolving 25% of the 
total conflicts. 

Concept 3: Automation Primary, Controller 
Secondary 

TFR pilots are equipped with a CSD with the 
RAT flight path replanning tool, but without CD&R 
tools. As in Concept 2, pilots are responsible for 
requesting and obtaining weather free re-routes, but 
are not responsible for resolving any conflicts. The 
autoresolver agent is responsible for resolving TFR- 
TFR and TFR-IFR conflicts, but as in Concept 2 it 
does not take weather into account. For TFR-IFR 
conflicts, the autoresolver agent preferentially 
modifies the route of the TFR aircraft. Controllers 
are equipped with a conflict probe tool and a conflict 
resolution tool and are responsible for IFR-IFR 
conflicts. TFR pilots can review route modifications 
datalinked to them by the autoresolver agent before 
executing them. TFR pilots use the CSD to generate 
and datalink reroute requests for weather avoidance 
to the Autoresolver but, if the reroute has conflicts, it 
will be automatically transferred to, and subsequently 
handled by, the controller. Through these allocations, 
pilots are not responsible for resolving any conflicts, 
ATC is responsible for resolving 25% of the total 
conflicts, and the auto-resolver agent is responsible 
for resolving 75% of the total conflicts. 

In all three concepts once the aircraft reached 
the top of descent, the controller became solely 
responsible for separation, while the pilots remained 
responsible for achieving a 105 second in trail 
separation by the final approach fix. The separation 
responsibility and equipage assignment are 
summarized in Table 1 below. 



Table 1. Equipages and responsibilities of pilots, controllers, and automation in each concept 


Flight 

Phase 

Cruise 

Descent/Arrival 

Concept 

1 

Pilots: Equipped with CD&R and interval 

management automation, responsible for TFR- 
TFR and TFR-IFR conflicts and spacing. 

Controllers: CD&R equipped and responsible for 
IFR-IFR conflicts 

Auto-resolver Agent: Responsible for no conflicts. 

Pilots: No CD&R, responsible for no 

conflicts. Equipped with interval 

management automation, responsible for 
spacing goals. 

Controller: CD&R equipped, responsible for 
all conflicts 

Concept 

2 

Pilots: Equipped with CD&R and interval 

management automation, responsible for spacing 
but not for conflicts. 

Controllers: CD&R equipped and responsible for 
TFR-IFR and IFR-IFR conflicts 

Auto-resolver Agent: Responsible for TFR-TFR 
conflicts. 

Pilots: No CD&R, not responsible for any 
conflicts. Equipped with interval management 
automation, responsible for spacing goals. 

Controller: CD&R equipped, responsible for 
all conflicts 

Concept 

3 

Pilots: Not CD&R equipped or responsible for any 
conflicts. Equipped with interval management 
automation, responsible for spacing goals. 

Controllers: CD&R equipped and responsible for 
IFR-IFR conflicts. 

Auto-resolver Agent: Responsible for TFR-TFR 
and TFR-IFR conflicts. 

Pilots: No CD&R, not responsible for any 
conflicts. Equipped with interval management 
automation, responsible for spacing goals. 

Controller: CD&R equipped, responsible for 
all conflicts 


weather, and perform a continuous descent approach 
III. Experiment Design (CD A) arrival from the top of descent (TOD) into 


To evaluate the impact of functional allocation Louisville Standiford Field Airport (SDF) using the 

concept used during cruise on system performance CBSKT-1 arrival shown top-down in Figure 1. 

during arrival, and on the workload and situation The simulated airspace, modeled after Kansas 

awareness of pilots and controllers during arrival, a C ity A ir Route Traffic Control Center (ZKC) and 
human-in-the-loop (HITL) simulation was Indianapolis Air Route Traffic Control Center (ZID), 
developed. The simulation airspace assumes a 3-time consists of two airspace “super-sectors” ZKC 90 and 

current day traffic density operating in both the ZKC 91, which are larger than current-day airspace, 

enroute cruise and arrival flight phases under the Super-sector ZKC90 was created by geographically 

three separation assurance functional allocation combining existing ZKC sectors 90 and 14, while 

concepts described above. “Super-sector” ZID 91 was created by combining the 

existing ZID sectors 91, 81 and 17. This airspace 
III. A. Scenario and Testing Design was then populated with additional aircraft to create a 

Traffic scenarios were created with aircraft 3-time current day traffic density. The traffic flow, 

being assigned an interval management clearance modeled after real traffic streams, encountered 

(designating lead aircraft and 105 sec in trail) prior to enroute weather cells west of ZKC 90, and then 

encountering convective weather in the cruise phase reached the TOD before merging and performing the 

of flight. They were then required to avoid the 

















CD A in ZID 91. The airspace sectors, weather cells, 


TOD, and merge point are illustrated in Figure 2. 



Figure 1. CBSKT-1 Arrival 
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Figure 2. Airspace and sector layout 


III.B. Independent Variables and Analysis 
Metrics 

A 3 (Concept: 1 - Pilot Primary, 2 - Controller 
Primary, 3 - Automation Primary) x 2 (Weather 
Complexity: Low, High) x 2 (Cockpit Weather 
Display: airborne radar, 3D NexRad) fully within- 
subjects factorial design was used. Specific numbers 
of TFR and IFR aircraft were designed into the 
scenarios to generate the following divisions of 
responsible conflicts count: 75% of conflicts to Pilots 
and 25% to Controllers for Concept 1; 75% of 
conflicts to Controllers and 25% to Automation for 
Concept 2; and 75% of conflicts to Automation and 
25% to Controllers for Concept 3. Low weather 
density had fewer and sparser weather cells, while 
high weather density had greater and denser weather 


cells. However, in the analysis of this paper, neither 
the Weather Complexity nor Cockpit Weather 
Display manipulations are examined, and will not be 
further mentioned. Twelve trials were conducted, 
one for each unique set of conditions. The CSD 
provided pilots with 3D and 2D spatial orientations 
of traffic information. 

Five types of data were collected to help 
quantify the effects of the function allocation during 
the arrival portion of the simulation: situation 
awareness data, aircraft trajectory data, voice data, 
workload data, and subjective ratings. Based on 
these data, a number of metrics were developed to 
quantify system performance and human factors 
performance. System performance, defined in terms 
of the system’s stability, efficiency, and compliance 







with safety, was measured by five key metrics: 1) 
throughput at the final approach fix; 2) aircraft 
kinetic energy; 3) number of violation of speed and 
altitude constraints; 4) Spacing interval variation at 
the merge point and at the final approach fix; and 5) 
variation in speed and altitude along the descent. The 
human factors performance was primarily measured 
by the workload and situation awareness ratings. 

III.C. Participants 

Eight ATP pilots and two air traffic controllers 
participated each week of the two-week experiment. 
However, only data collected during the second week 
of this experiment is reported in this paper, due to 
equipment failure and malfunction occurred during 
the first week of the experiment. The pilots’ flight 
time experience is summarized in Table 2 below [18]. 
Five of the eight second-week pilots were Captains 
and three were First Officers. Three pilots had 
previous experience flying CDAs but none of them 
had experience with merging and spacing operations. 
The two second-week controllers were retired, radar 
certified controllers, with 25 and 34 years of civilian 
air traffic control experience. 


Table 2. Pilot flight hours 


Total hours flown 
as a line-pilot 

N 

Total hours flown in 
“glass” cockpit 

N 

1-1000 

1 

1-1000 

4 

1001-3000 

0 

1001-3000 

1 

3001-5000 

4 

3001-5000 

3 

>5000 

3 

>5000 

0 


III.D. Simulation Equipment 

The HITL simulation used PC desktop-based 
single pilot and controller stations equipped with the 
Multi Aircraft Control System (MACS) and the 
Cockpit Situation Display (CSD) simulation software 
applications developed by NASA Ames Research 
Center’s Airspace Operations Laboratory (AOL) and 
Flight Deck Display Research Laboratory (FDDRL) 
respectively. MACS and CSD interfaces, shown in 
Figures 3 and 4, are connected and supported by a 
number of supplementary communication and 
networking applications. The MACS controller 
display is an emulation of current day controller 
stations with a ground-based automated conflict 
probe tool that can automatically detect and alert 



Figure 3. CSD interface display 



Figure 4. MACS interface Display 


controllers of impending conflicts. When part of their 
roles and responsibilities, controllers resolved 
conflicts, and resequenced aircraft, with a trial 
planner tool that they could use to graphically 
generate a new proposed trajectory, and then datalink 
the selected route modifications to the pilot. The 
CSD (shown in 2D mode in Figure 3) provided pilots 
with a display of traffic and weather, plus CD&R 
tools, flight path replanning tools, and interval 
management tools. The CSD provided an adjustable 
view of traffic, providing a 20-640 nm horizontal 
range, 2000 to 80000 feet vertical range, 
continuously adjustable 3D perspective views, and 
simulated airborne weather displays. The CSD could 
display all information in 2D (top down or profile) or 
in 3D views. The CSD also included an integrated 
trial planner, called the Route Assessment Tool 



(RAT). Similar to the controllers' trial planner, this 
tool allowed pilots to "grab" the current route and 
design new flight paths by stretching the route around 
weather. In Concepts 1 and 2 automated conflict 
alerting algorithms provided visual alerts when 
proposed routes created traffic conflicts. The RAT 
also provided feedback on how much delay the 
reroute generated. The CSD was integrated with the 
FMS allowing the pilot to execute the new route from 
the CSD. To alleviate controller workload, hand-offs 
and voice frequency changes were automated and 
check-ins were only required during merging and 
spacing operations. For controllers, IFR traffic was 
illuminated while TFR traffic was dimmed, unless in 
conflict with an IFR aircraft (in Concept 2). 

II LE. Distributed and Networked Simulation 

The simulation was distributed and networked 
over the internet with operators and/or participants 
located at different sites. Pilot stations and 
simulation management and other support stations 
were hosted in the FDDRL at the NASA Ames 
Research Center, while the two controllers and 
multiple ghost-controllers were located in the Center 
for Human Factors in Advanced Aeronautics 
Technologies (CHAAT) at California State 
University Long Beach (CSULB). Ghost-controllers 
controlled “ghost-sectors", which are airspace sectors 
adjacent to experimental sectors. Ghost sectors 
provide the airspace needed to facilitate the initiation 
and completion of hand-offs between sectors. Ghost- 
controller stations were operated by students and 
faculty at CSULB’s CHAAT. Multiple pseudo-pilots 
were located at the Systems Engineering Research 
Laboratory (SERL) at California State University 
Northridge (CSUN) and at the Human Integrated 
Systems Engineering Laboratory (HISEL) at Purdue 
University. Pseudo-pilot stations were operated by 
students and faculty at CSUN’s SERL and Purdue 
University’s HISEL. hi addition, online situation 
awareness and workload probes administered at 3- 
minute intervals asked both pilots and controllers to 
subjectively rate their workload and answer 
descriptive questions about their situation awareness. 
The readers are referred to [18-19, 21] for more 
details. 
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Figure 5. Throughput at CHRCL 

IV. Results and Discussion 

Measures were submitted to a 3 (Concepts) x 2 
(Display Type) repeated measures ANOVA via IBM 
SPSS 20.0. Greenhouse-Geisser adjustments were 
made for violations of sphericity when needed. The 
3*IQR outlier test was used for all variables. 
Outliers were replaced with the means of the design 
cells in which they occurred. As noted above, due to 
data-collection equipment failure and malfunction 
occurred during the first week of the experiment, 
only data collected during the second week of this 
experiment is reported in this section. 

IV.A. System Performance 

To assess the feasibility of the function 
allocation concepts, the performance of the system 
was examined with five key metrics: 1) throughput at 
the final approach fix; 2) aircraft kinetic energy; 3) 
spacing interval variation at the merge point and at 
the final approach fix; 4) variation in speed and 
altitude along the descent; and 5) number of violation 
of speed and altitude constraints. These metrics 
provide the basis for examining the system s 
efficiency, stability, robustness to the en route 
weather perturbation, and compliance with safety 
constraints. 

Throughput 

The throughput (in planes per hour) for each 
concept was measured at the final approach fix, 
CHRCL, and shown in Figure 5. The difference in 
the throughput was significant [F (2, 9) = 5.764, p < 
0.05]. The throughput of Concept 1 was higher than 
those of Concepts 2 and 3, and is very close to the 
target throughput of 34.3 planes/hour (105 sec in trail 
spacing) designed into the experiment. 


Aircraft kinetic energy 

The kinetic energy (KE) of the aircraft, shown in 
Figure 6, was calculated for the continuous descent 
approach (CDA) phase with the assumption that their 
masses are the same. In general, pilots were able to 
adhere to the target altitude profile but had trouble 
managing their speed. This translated into poor 
energy management as shown in Figure 6, where the 
energy is too low in Concept 2 and somewhat high in 
Concept 3. Concept 3 was found to have significantly 
higher kinetic energy than Concepts 1 and 2 from 
CBSKT to CHRCF. In Concept 3, pilots were able to 
decrease the aircraft energy at CHRCF to the same 
level as in Concept 1 . From SFEWW onward, 
Concept 2 was found to have the lowest kinetic 



Figure 6. Mean descent KE across Concepts 

The poor energy management can be further 
elucidated by examining mean KE target deviation 
shown in Figure 7. In all concepts, at the waypoint 
CBSKT, KE was severely off the target KE profile. 
This is especially significant for Concept 3, which 
had marginally higher deviation throughout the 
descent. Concept 2^s deviation drops significantly 
after CBSKT and remains below target until 
CHRCL. Concept 1 maintained least deviation from 
the target KE profile, particularly at CHRCL (AKE = 
0.23 MJ). 



Waypoint 


Figure 7. Mean KE deviation across Concepts 
Spacing 

Comparatively, the average spacing was 
significantly tighter in Concept 1 than in Concept 3 at 
all waypoints from CBSKT to CHRCL (p < .05), and 
closer to the target 105 seconds (See Figure 8). Mean 
spacing was significantly different between Concepts 
2 and 3 only at CBSKT. 



Figure 8. Mean Spacing across Concepts 

Altitude and Speed Profiles, and Constraint 
Violations 

For all waypoints along the descent, no 
significant differences in altitude were found across 
concepts. Altitude restriction violations were also 
calculated, but were relatively low for all concepts. 
Altitude target deviations did not exceed 80 feet for 
any waypoint in any concept. As illustrated in Figure 
9, altitude across concepts was managed in a 
consistent and stable manner. 



The means of the indicated airspeed (IAS) of the 
three concepts are shown in Figure 10. Along the 
CDA path from CBSKT to CHRCL (see Figure 1), a 
speed restriction is defined as a deviation of + 10 
knots from a waypoint’s speed constraint (except at 
CHRCL where there is no lower limit). Table 3 
shows the total violations at each waypoint with a 
breakdown of above and below deviations. Concept 
2 had the highest number of violations, most of 
which were violations of the lower limit. Concept 3 
had the second most violations, but these violations 
were mostly attributed to breaking the upper limit. 
These results agree with findings of higher mean IAS 
in Concept 3 and lower mean IAS in Concept 2, as 
shown in Figure 5. Analysis of IAS showed 
significant differences between Concepts 1 and 3 at 
CBSKT (F(1.133, 7.930) = 11.219, p < 0.01). Speed 
was higher throughout the descent from CBSKT to 
CHRCL for Concept 3 than for Concepts 1 and 2. 
Although aircraft were generally able to slow down 
and match the target profile by CHRCL, the 
consequence of carrying high energy did result in 
three upper limit speed violations at CHRCL in 
Concept 3, while both Concepts 1 and 2 had none. 
This highlights a critical shortcoming of Concept 3 in 
that it could not prevent more than 9% of the aircraft 
from violating an upper IAS limit at the final 
approach fix. Concept 2 matched Concept Ls speed 
profile at CBSKT, but distinctly fell below Concept 1 
at SLEWW (F(2, 14) = 54.005, p < 0.001). Concept 
2 fell below the target profile, and had low speeds at 
CHRCL. Based on number of violations, Concept 
Ls aircraft were able to match the target speed 
profile with better precision than the Concept 3 and 
Concept 2 aircraft. 



Figure 10. Mean descent IAS across Concepts 
Table 3. Number of IAS Restriction Violations 



Concept 1 

Concept 2 

Concept 3 

Above 

18 

15 

56 

Below 

37 

62 

4 

Total 

55 

77 

60 


IV.B. Situation Awareness and Workload 

The results on pilot and controller situation 
awareness and workload have been reported in a 
number of papers [18, 19, 21] and will only be 
summarized here. In general, it was found that pilots 
had the highest situation awareness in Concept 1 
(Pilot Primary), when they were most actively 
engaged in and responsible for separation assurance. 
Conversely, pilots showed lowest situation awareness 
when they were least engaged in separation assurance 
in Concept 3 (Automation Primary). Workload was 
more dependent on the flight phase than function 
allocation concept, but trends in multiple workload 
metrics indicated that pilots had lowest workload 
when they are responsible for resolving traffic 
conflicts. This finding suggested that, for pilots, 
increases in workload were driven more by gaining 
situational awareness than by higher separation 
assurance responsibility. Controllers experienced 
highest workload in Concept 2 (Controller Primary) 


where they had the highest level of responsibility, 
and lowest workload in Concept 3 (Automation 
Primary). Overall, both pilots and controllers 
indicated in post-simulation questionnaires that all 
three concepts were workable and that they are 
comfortable with the concepts. 

IV. C. Discussion 

Robustness to weather perturbation 

As shown in Figure 1, it was anticipated that the 
weather cells prior to the TOD and the merge point 
PRINC would disrupt the aircraft flow and create 
substantial variation in the initial conditions of the 
aircraft by the time they reach TOD or PRINC. 
However, previous analyses have shown that for all 
concepts there were no significant differences from 
the baseline profiles and no significant variation in 
terms of IAS, altitude, and KE at the TOD and the 
merge point PRINC [18]. That is, aircraft in each 
concept converged to a common flight state and had 
comparable initial conditions at the start of the CDA. 
The performance data from these previous 
examinations of this data (in terms of frequency with 
which aircraft entered and time spent in weather 
cells, time to pass weather and to reach TOD, the 
number of aircraft out of sequence or needing re- 
sequence, the number of spacing disengagement, and 
path stretch around weather cells) converged in 
showing that there was no significant differences in 
how pilots avoided the weather. The similar and 
comparable initial conditions at the TOD helped the 
aircraft tightly track the altitude profile without any 
statistically significant variations across all concepts 
as shown in Figure 9. Collectively these finding 
suggests that all concepts are robust to weather 
perturbation and performed comparably with one 
another. 

System Performance 

Based on the throughput and energy management and 
constraint violation results, pilot performance in 
Concept 1 (pilot primary) was superior to pilot 
performance in Concepts 2 (controller primary) and 3 
(automation primary). However, it is interesting to 
note that upon beginning the descent, the pilots are 
no longer responsible for separation assurance 
because this responsibility was assigned to the 
controller in all concepts. In light of the findings 
from [18] mentioned above, that all concepts had 
comparable flight states at TOD and at the merge 


point, this suggests that the significant differences in 
throughput and energy management and violation of 
constraints are most likely due to the different 
degrees of pilot “in-the-loop ’ involvement in 
managing the spacing and the flight profile during the 
CDA. With Concept 1, because pilots were assigned 
the separation responsibility before the CDA, they 
naturally would be actively involved in using the 
spacing tool to manage the spacing and in managing 
the descent even when they are no longer responsible 
for the separation. Consequently, they have the 
highest level of situation awareness among the 
concepts. With Concept 2, because the pilots had 
secondary responsibility in managing the separation 
before the CDA, they would be partially involved in 
managing spacing and the descent. Likewise, the 
controller was also not as actively involved in the 
loop because the spacing algorithm already handled 
the spacing between the aircraft and the controller 
only had to intervene when the spacing is projected to 
decrease below the minimum required separation. 
With concept 3, because the pilots were not at all 
involved in the separation prior to the CDA, they 
would further fall “out-of-the-loop” after the CDA. 
This is evidenced in the lowest situation awareness 
that pilots had for this concept. 


V. Conclusions 

A key research issue of great interest to NextGen 
researchers and developers is the design of viable 
separation assurance concepts. While the results of 
this study are applicable to the conditions designed 
and manipulated in the experiment, they have a 
number of implications with respect to the design of 
the allocation of separation assurance responsibility 
between pilots, controllers, and automation. First, 
although the operation was stressed with severe 
weather disruption, all three concepts were robust to 
this perturbation in the presence of 3-time higher than 
current day traffic density. In combination, the 
system performance data and situation awareness and 
workload data, point to a general conclusion that all 
concepts are workable for the pilots and controllers. 
Thus, in the design of NextGen operational concepts, 
this study suggests that these three function allocation 
concepts are viable design alternatives that deserve 
further study or consideration for integration with 
other NextGen concepts such as Trajectory Operation 
or Trajectory Based Operation [1]. In the context of 



these concepts, it would be essential to further 
explore system performance under a number of 
newly proposed TO/TBO system architectural 
elements such as required time performance, open 
and closed trajectories, and dynamic windows (i.e., 
allowed flexibility in 4D trajectory). These new 
elements and concepts form a paradigm to facilitate 
conformance monitoring both on the flight deck and 
on the ground, and present new challenges in terms 
of designing operation procedures and display 
systems for conformance monitoring and alerting 
under TO/TBO. 

Second, the results corroborate the fundamental 
automation design principle that the best form of 
automation design is one that involves the operator in 
the loop. This view is supported by this study, 
showing effects even though the required 
participation of the pilots was the same by the time 
they began the arrival phase. Because Concept l(i.e., 
pilot primary) requires pilots to be actively engaged 
“in-the-loop” with separation assurance, it appears to 
motivate the pilots to continue to maintain this 
involvement even when they are no longer required 
to. In contrast, because Concepts 2 (controller 
primary) and Concept 3 (automation primary) take 
the pilots out-of-the-loop to a greater extent than 
Concept 1, the performance under these concepts are 
inferior to Concept 1 . 
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